Field of the Invention
Embodiments of the present invention generally relate to refinement of disparity or depth images in three-dimensional (3D) image processing.
Description of the Related Art
Objects at different depths in the scene of a stereoscopic video sequence will have different displacements, i.e., disparities, in left and right frames of the stereoscopic video sequence, thus creating a sense of depth when the stereoscopic images are viewed on a stereoscopic display. The term disparity refers to the shift that occurs at each pixel in a frame between the left and right images due the different perspectives of the cameras used to capture the two images. The amount of shift or disparity may vary from pixel to pixel depending on the depth of the corresponding 3D point in the scene.
In many stereo vision applications, it is important to know the depths of objects in a scene. The depth information for a stereo frame or image is typically computed from the disparities between the pixels in the left image and corresponding pixels in the right image because depth is proportional to the reciprocal of the disparity. These disparities are typically communicated in the form of a disparity map or image that records the disparity of each pixel as a horizontal shift amount between corresponding pixels in the two images. To determine the disparities, a stereo matching algorithm, also referred to as a stereo correspondence algorithm is used.
The computation of stereo correspondence between a left-right image pair typically results in some pixel matches that are erroneous or ambiguous due to factors such as the inherent imprecision in the measurement of light intensity of the imaging systems capturing the stereo images, the presence of low or no texture objects in the scene, occlusion, background video, etc. Errors and ambiguity in matching occur even when stereo matching algorithms are used that consider the factors that can introduce error and ambiguity during the match computation. The errors and ambiguity are manifested as noise in the disparity image. Therefore, in many applications, the disparity image is refined to attempt to detect and suppress the noise. Common refinement techniques operate on a per-pixel basis, using confidence scores and thresholds to detect and suppress errors. Such per-pixel techniques may suppress ambiguous yet otherwise accurate measurements by error.